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CONTROLLER FOR PHOTOGRAPHING APPARATUS AND 
PHOTOGRAPHING SYSTEM 
BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to a controller 
for a photographing apparatus and a photographing 
system with a high operational characteristic and a 
high visibility suitable for a photographing operation 
of the apparatus that is disposed at a remote place and 
that is used for a monitoring operation, an observing 
operation, a guiding operation, a presenting operation, 
and so forth. 

Description of the Related Art 

As shown in Fig. 23, when the user controls a 
photographing apparatus disposed at a remote place, he 
or she operates a pan tilter in eight directions (up, 
down, left, right, upper right, lower right, upper 
left, and lower left directions) with eight-direction 
keys, a zooming controller, and a wide-angle controller 
so as to photograph a desired object while observing a 
photographed picture 6A on a monitor 2. In the 
structure shown in Fig. 23, the user moves a cursor 7 
to one of the direction ke^s 10 with a mouse 8. 
Alternatively, after the user has controlled a 
photographing apparatus disposed at a remote place in 
the above-described method and registered pan tilter 
information and zoom information of positions of 



pictures to be photographed, he or she drives the 
photographing apparatus at absolute positions 
corresponding to the registered positions so as to 
select pictures. 

In the conventional controller, a picture 
that is displayed on the monitor is limited in the 
range of which the photographing apparatus is moved by 
the pan tilter. Thus, when the user photographs a 
desired object, he or she should operate the pan tilter 
in the full range thereof. Consequently, the user 
should have skill in operating the pan tilter. 

When the user changes the photographing 
direction with the conventional direction keys, even if 
he or she stops pressing the direction keys, since the 
pan tilter does not immediately stops and thereby he or 
she may not catch a desired object. When the direction 
varying speed of the photographing apparatus with the 
pan tilter is low, although such a problem may be 
solved, since the response characteristic deteriorates, 
a high operational characteristic cannot be obtained. 

When the user wants to place a desired object 
at the center of the angle of view of the photographing 
apparatus, since he or she- controls the photographing 
direction while observing a picture on the monitor, he 
or she should determine the photographing direction on 
trial and error basis. Thus, the user may spend a long 
time for controlling the photographing apparatus. 



Moreover, to properly operate the photographing 
apparatus, the user should have skill. 

When picture and control information is 
exchanged with a photographing apparatus disposed at a 
remote place through a low-capacity network, the 
control information may be lost and/or picture 
information may be delayed due to an irregularity of 
their arrival intervals. If the pan tilter or the 
zooming controller is operated for picture and control 
information that has been delayed or lost, even if the 
user causes the pan tilter and the zooming controller 
to place the object at the desired position, the pan 
tilter and the zooming controller do not properly 
operate. Thus, the object is placed at an improper 
position due to the delay. In addition, depending on 
the line condition, the arrival intervals of picture 
information vary. Thus, the user should control the 
pan tilter and the zooming controller based on a 
prediction. Consequently, the user cannot properly 
control the pan tilter and the zooming controller. 

OBJECTS AND SUMMARY OF THE INVENTION 
Therefore, an object of the present invention 
is to provide a controller for a photographing 
apparatus for allowing the user to designate a desired 
position or a desired area on a panorama picture 
displayed as a part or all the moving range of a pan 



tilter so that the user can easily obtain a desired 
picture with the photographing apparatus. 

Another object of the present invention is to 
provide a controller for a photographing apparatus and 
a photographing system with a high visibility and a 
high operational characteristic that allow the user to 
designate a desired position or a desired area on a 
screen and select an object with the designated 
position or area and the photographing apparatus to 
place the selected object at the center of the screen. 

A first aspect of the present invention is a 
controller for a photographing apparatus having a 
photographing portion with driving means that allows 
the photographing direction of photographing means, to 
be varied, comprising a displaying means for displaying 
a panorama picture generated with a picture 
photographed by the photographing means, and a 
controlling means for referencing the panorama picture 
and varying the photographing direction of the 
photographing means. 

A second aspect of the present invention is a 
controller for a photographing apparatus having a 
photographing portion with- driving means that allows 
the photographing direction of photographing means to 
be varied, the controller comprising an operation area 
in which a panorama picture generated with a picture 
photographed by the photographing means is displayed, 



and a picture selecting means for allowing the user to 
designate a desired point in the operation area, 
selecting an object photographed by the photographing 
means corresponding to the designated point, and moving 
the selected object to desired positional coordinates 
of the driving means . 

A third aspect of the present invention is a 
controller for a photographing apparatus having a 
photographing portion with driving means that allows 
the photographing direction of photographing means to 
be varied, the controller comprising an operation area 
in which a panorama picture generated with a picture 
photographed by the photographing means is displayed, 
and a picture selecting means for allowing the user to 
designate a desired area in the operation area, 
selecting an object photographed by the photographing 
means corresponding to the designated area, and moving 
an object at the position corresponding to a desired 
point generated with the desired area to desired 
positional coordinates of the driving means. 

A fourth aspect of the present invention is a 
photographing system having a photographing portion 
with driving means that allows the photographing 
direction of photographing means to be varied and a 
controller for a photographing apparatus, the 
controller controlling the photographing portion, 
wherein the controller comprises an operation area in 



which a panorama picture generated with a picture 
photographed by the photographing means is displayed, 
and a picture selecting means for selecting an object 
photographed by the photographing means in the 
operation area and moving the selected object to 
desired positional coordinates of the driving means. 

A picture photographed by a pan tilter camera 
that is disposed at a remote place and that can be 
moved in various directions is sent to a computer. The 
picture is displayed as a panorama picture in a display 
area of a monitor. The direction of a picture 
selecting means corresponding to the direction of an 
object to be placed at the center of the angle of view 
of the photographing apparatus in the panorama picture 
is designated by a pointing device connected to the 
computer!" Since the pan tilter' is controlled with 
reference to the panorama picture, a desired picture 
can be photographed by the photographing apparatus . 

In addition, the environment of the place at 
which the pan tilter camera is disposed is displayed as 
a panorama picture in the panorama operation area of 
the monitor of the computer. A desired point to be 
placed at the center of thb angle of view of the 
photographing apparatus in a picture of the panorama 
operation area or a desired point generated with a 
desired area is designated by the pointing device 
connected to the computer. Thus, in the method of 



which the result is input, a selected object can be 
easily placed at the center of the screen. In 
addition, since a desired point in the operation area 
on the screen or a desired point generated with a 
desired area is designated with the pointing device, 
the user can easily know the driving direction of the 
pan tilter camera. In addition to the panorama 
operation area, another operation area for a picture 
may be displayed. 

These and other objects, features and 
advantages of the present invention will become more 
apparent in light of the following detailed description 
of a best mode embodiment thereof, as illustrated in 
the accompanying drawings . 

~" BRIEF DESCRIPTION OF THE DRAWINGS ? 
Fig. 1 is an external view for explaining a 
system according to an embodiment of the present 
invention; 

Fig. 2 is a schematic diagram for explaining 
a screen of a monitor according to the embodiment of 
the present invention; 

Fig. 3 is a bloctf: diagram showing the 
structure of the system according to the embodiment of 
the present invention; 

Figs. 4A to 4F are schematic diagrams for 
explaining a method for generating a panorama picture 



according to the embodiment of the present invention; 

Figs. 5A to 5D are schematic diagrams for 
explaining a method for generating a panorama picture 
according to the embodiment of the present invention; 

Figs. 6A to 6C are schematic diagrams for 
explaining a method for generating a panorama picture 
according to the embodiment of the present invention; 

Figs. 7A and 7B are schematic diagrams for 
explaining a method for generating angular information 
of a pan tilter camera with positional coordinates in a 
panorama operation area according to the embodiment of 
the present invention; 

Figs. 8A and 8B are schematic diagrams for 
explaining a plane - spherical surface converting 
method according to the embodiment of the present 
invention; - - _ ^ — . 

Figs. 9A and 9B are schematic diagrams for 
explaining a coordinate converting method in the 
operation area according to the embodiment of the 
present invention; 

Figs. 10A to IOC are schematic diagrams for 
explaining a coordinate converting method in the 
panorama operation area according to the embodiment of 
the present invention; 

Figs. 11A and 11B are schematic diagrams for 
explaining positional information and angular 
information of a pan tilter camera according to the 
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embodiment of the present invention; 

Figs. 12A and 12B are schematic diagrams for 
explaining angular coordinates of the pan tilter camera 
and positional coordinates in the panorama operation 
area according to the embodiment of the present 
invention; 

Figs. 13A to 13D are schematic diagrams for 
explaining the angle of view of the pan tilter camera 
and a frame in the panorama operation area according to 
the embodiment of the present invention; 

Fig. 14 is a graph for explaining a 
conversion method of zoom data and magnification data 
according to the embodiment of the present invention; 

Fig. 15 is a flow chart showing an example of 
the overall process according to the embodiment of the 
present invention; — 

Figs. 16A and 16B are flow charts showing an 
example of the process of a timer event according to 
the embodiment of the present invention; 

Fig. 17 is a flow chart showing an example of 
the process of a mouse moving event according to the 
embodiment of the present invention; 

Fig. 18 is a flo^ chart showing an example of 
the process of a mouse button down event according to 
the embodiment of the present invention; 

Fig. 19 is a flow chart showing another 
example of the process of a mouse button down event 



according to the embodiment of the present invention; 

Fig. 20 is a flow chart showing an example of 
the process of a mouse up/down event according to the 
embodiment of the present invention; 

Fig. 21 is a schematic diagram showing the 
structure of a system according to a second embodiment 
of the present invention; 

Fig. 22 is a block diagram showing the 
structure of the system according to the second 
embodiment of the present invention; and 

Fig. 23 is a schematic diagram for explaining 
a controller for a photographing apparatus. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Next, with reference to the accompanying 
drawings, embodiments of the present invention will be 
described. Fig. 1 shows an outline of the structure of 
a system according to a first embodiment of the present 
invention. A monitor 2 and a mouse 8 are connected to 
a computer 1. The computer 1 controls the driving 
operation of a pan tiler camera 3 disposed at a remote 
place. In other words, a controller for the 
photographing apparatus is, v composed of the computer 1. 

The pan tilter camera 3 is integrally 
composed of a pan tilter portion and a camera portion. 
In Fig. 1, the pan tilter camera 3 is disposed on a 
real scene as denoted by 4. A screen of a picture 



photographed by the pan tilter camera 3 is denoted by 
5. This screen is hereinafter referred to as a 
photographed screen. The photographed screen 5 is an 
actually photographed screen. When a zoom lens of the 
pan tilter camera 3 is placed on the telephotograph 
side, the angle of view decreases. In contrast, when 
the zoom lens of the pan tilter camera 3 is placed on 
the wide-angle side, the angle of view increases. 

A picture on a photographed screen 5 captured 
by a pan tilter camera 3 is sent to a computer 1 
through a video cable or the like. The picture data 
sent to the computer 1 is decoded and displayed on a 
monitor 2. The monitor 2 displays the photographed 
screen 5 in an operation area 6A on the monitor 2. A 
panorama picture including the picture photographed by 
the pan tilter- camera 3 is displayed in a panorama 
operation area 6B. An arrow-shaped cursor 7 is 
displayed at the position of a mouse pointer of a mouse 
8 in the operation area 6A or the panorama display area 
6B. The user designates a desired point or a desired 
area in the operation area 6A or the panorama operation 
area 6B with the mouse 8\so as to operate the pan 
tilter camera 3. In the panorama operation area 6B, a 
frame 6C that represents the current position and the 
angle of view of the pan tilter and a pan tilter 
limiter 6D are superimposed to the panorama picture. 
The pan tilter limiter 6D represents the moving range 
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of the pan tilter camera. In addition, when necessary, 
a panorama generation button 6E is displayed on the 
monitor 2 . 

As shown in Fig. 2, the operation area 6A and 
the panorama operation area 6B are displayed on the 
monitor 2. With the mouse 8, the user can move the 
cursor 7 and designate a desired point or a desired 
point generated with a desired area in the operation 
area 6A or the panorama operation area 6B. The user 
operates the pan tiler so that an object corresponding 
to the designated point is placed at the center of the 
operation area 6A. In other words, when the user 
inputs a result to be displayed, an object selected 
corresponding to the input data is displayed at the 
center of the operation area 6A. 

Fig. 3 is a- block- diagram showing the overall 
system according to the embodiment of the present 
invention. The system shown in Fig. 3 comprises a 
camera portion 11, a pan tilter portion 12, a TV 
monitor 13, a computer 1, a pointing device 14 (such as 
a mouse 8), and a monitor 2. The pan tilter camera 3 
comprises a camera portion 11 and a pan tilter portion 
12. For example, the cameta portion 11 is disposed on 
the pan tilter portion 12. The camera portion 11 
comprises a lens block portion 15, a zoom lens 16, a 
zoom portion 17, a zoom lens motor 18, a solid state 
image pickup device 19, a signal separating/automatic 
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gain adjusting circuit. (SH/AGC) 20, an A/D converter 
21, and a signal processing circuit 22. The camera 
portion 11 represents a video camera. 

The pan tilter portion 12 comprises a mode 
controller 23, a camera controller 24, a pan tilter 
controller 25, a pan motor 26, a tilt motor 27, and a 
pan tilter 28. The computer 1 comprises a controlling 
portion 31, a video capture portion 29, and a storing 
portion 30. The video capture portion 29 is composed 
of a video capture board. 

Rays emitted from an object are focused to 
the solid state image pickup device 19 through a lens 
set and a diaphragm of the lens block portion 15. An 
example of the solid state image pickup device 19 is a 
CCD (Charge Coupled Device). The focused rays (field 
picture ) are converted into a picture signal- and then — - 
sent to the signal separating/automatic gain adjusting 
circuit 20. The signal separating/automatic gain 
adjusting circuit 20 samples/ holds the picture signal 
and controls the gain of the picture signal with a 
control signal of an auto iris (AE) . The resultant 
picture signal is sent to the signal processing circuit 
22 through the A/D converter 21. The signal processing 
circuit 22 converts the received picture signal into a 
brightness signal (Y) , a color signal (C) , and a video 
signal and sends these signals as picture signals to 
the TV monitor 13 and the video capture portion 29. of 
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the computer 1. 

The lens block portion 15 of the camera 
portion 11 drives the zoom lens 16 and thereby varies 
the angle of view of an object to be photographed. The 
lens block portion 15 causes the zoom lens motor 18 
that is for example a stepping motor to rotate, thereby 
driving the zoom- lens 16 corresponding to a drive 
command received from the camera controller 24 of the 
pan tilter portion 12. The camera controller 24 
performs a lens controlling operation (for example, 
focusing operation and zooming operation) , an exposure 
controlling operation (for example, diaphragm 
controlling operation, gain controlling operation, and 
speed controlling operation of electronic shutter), 
white balance controlling operation, a picture quality 

controlling operation,- -and -so forth of the camera — - 

portion 11. In addition, the camera controller 24 
interfaces with the mode controller 23. As interface 
controlling operations with respect to the zoom lens 
16, the camera controller 24 sends a control signal to 
the motor driver corresponding to a drive command of 
the zoom lens 16 received from the mode controller 23 
so that the zoom lens 16 i£ placed at the position 
designated by the command. In addition, the camera 
controllers 24 always sends positional information of 
the zoom lens 16 to the mode controller 23. 

The camera portion 11 is disposed on the pan 
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tilter portion 12 that has a degree of freedom that are 
rotating directions of two axes of pan and tilt. The 
pan tilter portion 12 causes the pan motor 26 and the 
tilt motor 27 to rotate corresponding to a drive 
command received from the pan tilter controller 25, 
thereby driving a pan head and a tilt head of the pan 
tilter 28. The motors 26 and 27 are composed of for 
example stepping motors. The pan tilter controller 25 
sends a control signal to the motor drivers so that the 
pan head and the tilt head are driven to positions 
corresponding to a pan drive command and a tilt drive 
command received from mode controller 23. In addition, 
the pan tilter controller 25 always sends positional 
information of the pan head and the tilt head to the 
mode controller 23. 

The mode controller 23 controls the overall 
system corresponding to the internal states of the 
camera portion 11 and the pan tilter portion 12 and the 
interface information received from the outside of the 
pan tilter camera 3 as will be described later. The 
mode controller 23 is connected with for example the 
computer 1 and RS-232C interface. The mode controller 

23 sends drive commands received from the computer 1 to 
the pan tilter controller 25 and the camera controller 

24 so as to drive the pan tilter 28 and the zoom lens 
16 of the lens block portion 15. In addition, the mode 
controller 23 sends current positional information 
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received from the pan tilter controller 25 and the 
camera controller 24 to the computer 1; 

According to the embodiment, the computer 1 
is used to select a picture photographed by the pan 
tilter camera 3. The computer 1 processes graphics in 
the operation area 6A and the panorama operation area 
6B displayed on the monitor 2 and information of a 
designated position and a clicking operation of the 
pointing device 14 (mouse 8) and sends the resultant 
data to the mode controller 23. To display a picture 
photographed by a camera portion 11 on the monitor 2, a 
video capturing portion 29 is u sed - The video 
capturing portion 29 allows a video signal received 
from the camera portion 11 to be displayed on the 
monitor 2 with a desired picture quality. In addition, 

the video capturing portion 2 9 allows a picture to be 

captured in a particular picture format (for example, 
bit map format, still picture JPEG format, moving 
picture JPEG format, or the like) with a particular 
picture quality and to be stored in the storing portion 
30 (for example, a hard disk) of the computer 1. 

Next, with reference to Fig. 4, an example of 
a method for generating a panorama picture displayed in 
the panorama operation area 6B will be described. It 
should be noted that according to the present 
invention, a panorama picture may be generated by 
another method. When the panorama generation button 6E 



is pressed, a panorama picture is generated. 

Now, it is assumed that the environment in 
the place where the pan tilter camera 3 is disposed is 
a spherical surface. The spherical surface is referred 
to as virtual spherical surface. In Figs. 4A to 4F, 
two adjacent pictures on the virtual spherical surface 
are combined to one panorama picture. To generate a 
panorama picture, as shown in Fig. 4A, the pan tilter 
camera 3 disposed at the center of the sphere 
photographs two adjacent pictures on the virtual 
spherical surface. The pan tilter camera 3 photographs 
a plane perpendicular to the optical axis of the lens 
thereof. Fig. 4D shows a situation of which two 
adjacent pictures on the virtual spherical surface are 
photographed by the pan tilter camera 3 and the two 
-pictures are mapped to the plane perpendicular to~ the 
, optical axis. When two adjacent pictures are simply 
: combined, they overlap and distort at the overlapped 
. portion. 

To prevent two adjacent pictures from 
overlapping and distorting, they are mapped to the 
virtual spherical surface as shown in Fig. 4B. Fig. 4E 
shows a situation of which- two photographed pictures 
that are planes perpendicular to the optical axis are 
mapped to the virtual spherical surface. In such a 
manner, planes perpendicular to the optical axis 
(namely, photographed pictures) are mapped to the 
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virtual spherical surface. The mapped pictures are 
combined in such a manner that an overlapped portion 
and an unnecessary portion are removed. The picture 
mapped on the virtual spherical surface is normalized 
with longitude and latitude. Thus, a panorama picture 
as shown in Figs. 4C and 4D is generated. 

Next, a method for generating a panorama 
picture will be described. In this method, as shown in 
Figs. 5A to 5D, one panorama picture is generated by 
combining 10 pictures. First, the pan filter camera 3 
(not shown) disposed at the center of the sphere 
photographs 10 pictures. At this point, as shown in 
Fig. 5A, by matching the optical axis of the lens of 
the pan filter camera 3 to positions denoted by 
circles, the pan filter camera 3 can obtain pictures 1 
to- 10.- -As shown- in- Fig.- 5Br the pictures-photographed 
by the pan filter camera 3 are pictures,, on the plane 
perpendicular to the optical axis of the lens. The 
obtained pictures are mapped to the virtual spherical 
surface. Thereafter, as shown in Fig. 5C, the pictures 
are normalized with latitude and longitude. The 
pictures are obtained in such a manner that they are 
smoothly combined without a break. Thereafter, an 
overlapped portion and unnecessary portion are removed. 
Thus, a panorama picture of which 10 picture are 
smoothly combined is generated. 

Next, with reference to Fig. 6, another 
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method for generating a panorama picture will be 
described. In this method, pixels obtained by the pan 
tilter camera 3 are designated to pixels of a panorama 
picture normalized with latitude and longitude (namely, 
coordinates (s, t) ) . As in the method shown in Figs. 
5A to 5D, when pixels of pictures photographed by the 
pan tilter camera 3 are designated to pixels of a 
panorama picture, part of pixels of the panorama 
picture may not be designated. All pixels of pictures 
photographed by the pan tilter camera 3 should be 
designated to pixels of the panorama picture. The 
panorama picture is composed of pixels calculated for 
individual coordinate points in the following process. 
Angular coordinates (a, 3) (see Fig. 6B) on the virtual 
spherical surface corresponding to coordinates (s, t) 
(see Figr 6) * of a panorama picture are calculated"""^^ 
corresponding to Eq. (1). 

(a, JJ) - (a(s)V b(t) ) - 5 . . . . (1) 

(Eq. (1) will be described later with reference to Fig. 
7A and 7B.) 

As shown in Fig. 6C, coordinate data (£, rj) 
of the obtained picture is calculated with the 
coordinates (s, t) , the angular coordinates (0, 4>) of a 
pan tilter 28, and photographing magnification assuming 
that the wide edge of the photographing apparatus is 
defined as one magnification corresponding to Eq. (2). 

(£>, n) = (f(a, 3, 0, <j>, y)/ g(«, 3/ 0, <t>, y) ) 



... (2) 

(Eq. (2) will be described later with reference to 
Figs . 8A and 8B. ) 

Corresponding to the above-described 
equations, pixels of the panorama picture are 
correlated with obtained pictures so as to generate a 
combined picture (namely, the panorama picture) . 

Next, with reference to Figs. 7A and 7B, a 
method for converting coordinates (s, t) of a panorama 
picture into angular coordinates (a, 3) on the virtual 
spherical surface will be described. In Fig. 7A, 
PragMin represents angular data at the left edge 
assuming that the home position of the pan tilter 28 
(for example, the center in the moving range of the pan 
tilter 28) is 0 (rag). PragMax represents angular data 
at- the right edge assuming that the home-position^of^ 
the pan tilter 28 is 0 (reg) . Ny 2 represents a 
horizontal coordinate of the panorama operation area- 
SB. -Ny 2 / 2 represents coordinate data at the right 
edge of the panorama operation area 6B. 

To obtain the pan angle oc with the coordinate 
data s, since the following relation is satisfied 

(PragMax - a) : fPragMax - PragMin) .= (Ny 2 / 
2 - s) : Ny 2 

the pan angle a is expressed as follows. 

a = PragMax - (PragMax - PragMin) x (Ny 2 / 2 

- s) / Ny 2 

20 



In Fig, IB, TragMin represents angular data 
at the upper edge assuming that the home position of 
the pan tilter 28 is 0 (rag) . TragMax represents 
angular data at the lower edge assuming that the home 
position of the pan tilter 28 is 0 (rag) . Nz 2 
represents a vertical coordinate of the panorama 
operation area 6B . -Nz 2 / 2 represents coordinate data 
at the upper edge of the panorama operation arae 6B. 
Nz 2 / 2 represents coordinate data at the lower edge of 
the panorama operation area 6B. 

To obtain the tilt angle 3 with the 
coordinate data t, since the following relation is 
satisfied, 

(TragMax - 3) : (TragMax - TragMin) = (Nz 2 / 
2 - t) : Nz 2 

the tilt angle 3- is- expressed as follows . - — ... 

3 = TragMax - (TragMax - TragMin) x (Nz 2 / 2 

- t) / Nz 2 

Next, with reference to Figs. 8A and 8B, the 
method for converting a plane into a spherical surface 
will be described. As shown in Fig. 8A, the spatial 
coordinates of a point (£, n) of a photographed picture 
orienting the home positiofi (the origin of latitude and 
longitude) are expressed as follows. 

P = 6 X + k^e, + k 2 r)6 n 
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At this point, the following relations are 

satisfied. 

k x = tan (X / 2y) / (Ny / 2) 

k 2 = tan (p / 2r) / (Nz / 2) 
where (Ny, Nz) represent the drive ranges (y direction 
and z direction) of the mouse pointer of the pointing 
device 14 (mouse 8); (A, \x) represents the horizontal 
angle of view and vertical angle of view at the wide 
edge; and y represents the current zoom relative 
magnification (magnification information) assuming that 
the wide edge is one time (x 1) . 

In addition, as shown in Fig. 8B, a three- 
dimensional rotation matrix is generally expressed as 
follows . 
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Since the direction of one point (£, n) of a 
photographed picture that is panned and tilted by 
angular information (0, <J>) from the home position is 
the same as the direction of one point (a, 3) apart 
from the home position, the following relation is 
satisfied. 

R z (9) R y (4>) p = lR z (a) (|3) e x 
When the formula is solved with respect to p, 
the following relation is satisfied. 

p = IR y (-<D) R x (a - 6) Ry (p) e x 
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= / 



cos(a - 8) cos <J> cos fi + sin ^ sin p 
sm(a-0)cosj8 

- cos(a - 9) sin^ cos/? + cos^ sin 0 
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.. (3) 



Thus, ^ and n are obtained as follows. 
1 = 1. /a 

S = -lb / k x = -b / k la 

n = lc / k 2 = c / k 2a 
With the above formula, (£, n) projected to the 
photograph coordinates can be obtained with coordinate 
data with an angle (a, 3) from the home position. 



. 23 



S = (-sin (a - 9) cos 3) / (k x (cos (a - 0) 
cos <J> cos 3 + sin <J> sin 3)) 

n = (-cos (a - 0) sin <J> cos 3 + cos <p sin 3) 
/ (k 2 (cos (a - 0) cos <J) cos 3 + sin <t> sin 3)) 

5 Coordinate data (£, ri) on the obtained picture by the 

pan tilter camera 3 can be obtained from angular 
coordinates (a, 3) on the virtual spherical surface 
corresponding to coordinate (s, t) of a panorama' 
picture. Thus, a panorame picture can be generated. 

0 In contrast, coordinate data with an angle 

(a, 3) can be obtained with (£, ri) projected to 
photograph coordinates corresponding to the following 
formula. 

Since . 1 = |p| 

5 a = 1 / 4 (1 + k x 2 ^ 2 + k 2 V) 
; b __ k ^ y^y - (1 v 

c = k 2 ri / y/ (1 + kj 2 ^ 2 + k 2 2 r| 2 ) .* -•.-....,•„.-. ^. 
-'^■•^ff wher^'V ( ) 'represents that ; the square root of the 

calculated result in ( ) is obtained. 
:0 Form Formula (3), the following relations are 

satisfied-. , 

* a = cos (a - 0) - cos <J) cos 3 + sin $ sin 3 

b = sin (a - 0) cos 3 

c = -cos (a - 0) sin <t> cos 3 + cos <p sin 3 
!5 Thus, the following relations are satisfied. 

a sin $ + c sin 0 = sin 3 

tan (a - 9) = b / (a cos <p - c sin 0) 



Thus, the following relations are satisfied, 

3 = sin' 1 (sin <j> / V (1 + k x 2 C + k 2 2 n 2 ) + sin 

e k 2 n / V(i + k 2 2 + k 2 2 n 2 ) 

a = tan" 1 (-k 2 5 / (cos 0 - k 2 n sin 9) ) + 9 
Thus, the pan angle a and the tilt angle 3 
can be obtained as follows. 

(a, 3) = (f(£, n, 9, <j>, y) / g(S, ri/ e, 0, y) ) 

... (4) 

If an error is permitted to some extent, (a, 
3) can be expressed as follows. 

a = 9 + (A / Y ) x (5 / Ny) 

3 = 0 + (]i / y) x (n / Nz) 
In other words, Eq. (4) can be simplified as follows. 

(a, 3) = (f(^, 9, y)/ g(H, <t>/ Y) ) ... (5) 

Next, with reference to Fig. 9, a method for 
calculating angular- information^ (a,~ 7 3) of the pan — ^ 
tilter 28 expressed b y ; Eq. ; (4) , and Eq. J (5) ^with^^i^i 
positional coordinates (£, n) of the . operation area 
will be described. First of all, an example of a 
method for directly designating a desired point in the 
operation area 6A will be described. Assuming that the 
center of the operation area 6A is defined as (0, 0) of 
relative coordinates as shown in Fig. 9A, the 
positional coordinates (£, n) of the mouse pointer of 
the mouse 8 in the operation area 6A are obtained. 

Next, another method for designating a 
desired point generated with a desired area in the 
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operation area 6A will be described. As shown in Fig. 
9A, after a start point (ml, nl) in a desired area is 
designated, an end point (m2, n2) in the desired area 
is designated. As the coordinates at the center of the 
rectangle generated with these two points, a desired 
point (£, ri) is obtained as Eq. (6). 

U/ n) = ((ml, nl) + (m2, n2)) / 2 ... (6) 

Fig. 9A shows coordinates of the mouse 8 
(pointing device 14) in the operation area 6A. In Fig. 
9A, the moving range (y direction and z direction) of 
the mouse pointer of the mouse 8 in the operation area 
6A is denoted by (Ny x , NzJ . Angular coordinates (a, 
P) of the pan tilter 28 are obtained with positional 
coordinates (£, n) of the desired point (at the mouse 
pointer of the mouse 8), angular information (9, <t>) 
that " represents the ^orien^ 

and magnification information (y) of the current zoom - 
relative magnification assuming that the wide edge of 
the zoom lens 16 is defined as one magnification 
corresponding to Eq. (4) or Eq. (5). 

The angular coordinates (a, 3) shown. in Fig. 
9B are used to place a position designated by the 
pointing device to the center of the photographed 
screen assuming that the home position of the pan 
tilter 28 is defined as the origin of latitude and 
longitude. 

The coordinates obtained in Figs. 9A and 9B 



may be absolute coordinates of the screen of the 
monitor 2 or relative coordinates assuming that the 
center of the operation area 6A is defined as (0, 0) . 
In the coordinates shown in Figs- 9A and 9B, 
coordinates in the pan direction are represented by 
ml, m2, 9, and a and coordinates in the tilt direction 
are represented by r\, nl, n2, <p, and (3. 

Thus, when the mouse pointer of the mouse 8 
is present in the operation area 6A, the angular 
information (a, 3) of the pan tilter 28 is calculated 
with the angular information (9, <p) of the current pan 
tilter 28 obtained with received data, the zoom 
magnification information (y) / and the positional 
information (£, ri) at the mouse pointer of the mouse 8 
corresponding to Eq. (4) or Eq. (5) so that the 
" designated ob j ect is placed" at the^¥nte"r^6f^he "" T '~""* ~ ^^-r^^^-- 
operation area 6A. The angular . coordinates (a, < (3) » of - 
the pan tilter 28 are ' converted? inter internal - : ±t^$:V'^ : i< ^ - '^>M^r;t"^ • 
positional information (PNew, TNew) as shown in Figs. 
11A and 11B. The resultant internal positional 
information (PNew, TNew) is stored in a send buffer 
along with an absolute position drive command of the 
pan tilter 28. In addition, as will be described 
later, a data send request flag (FlagSo) is set so that 
data is sent upon occurrence of a timer event. 

Next, with reference to Figs. 10A, 10B, and 
10C, a method for converting positional coordinates (5, 
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ri) of the mouse pointer of the mouse 8 in the panorama 
operation area 6B of the panorama picture into angular 
coordinates (a, 3) corresponding to the present 
invention will be described. As with the method for 
5 directly designating a desired point in the operation 

area 6A, as shown in Fig. 10A, with a method for 
directly designating a desired point in the panorama 
operation area 6B, positional coordinates (£, ri) at the 
mouse pointer of the mouse 8 can be obtained. 

10 Next, another method for designating a 

desired point generated with a desired area in the 
panorama operation area 6B will be described. As shown 
in Fig. 10A, after a start point (ml, nl) of a desired 
area is designated, the end point (m2, n2) of the 

15 desired area are designated. Corresponding to Eq. (6), 

— ~a desired point' ~ ( £7 " n )~~is obtainedT"^ ..y ~ 1 - ^ ? ^ F " 

.^u-... In Fig. 10A, the moving range (y direction ■^^■^^^&&'i~*--'<^ : --*->«*& 

and z direction) of the mouse pointer of the mouse 8 in 
the panorama operation area 6B (the moving range is 

20 defined as the coordinates of the mouse pointer of the 

mouse 8 (pointing device 14) in the panorama operation 
area 6B) is represented by (Ny 2 , Nz 2 ) . The moving 
range is limited by the pan tilter limiter 6D denoted 
by dotted lines in the panorama operation area 6B. The 

25 pan tilter limiter 6D represents the moving range of 

the optical axis of the lens of the pan tilter camera 
3. In other words, a point cannot be designated out of 
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the pan tilter limiter 6D. Positional coordinates (x, 
y) in the panorama operation area 6B, angle-of-view 
information (s, t), and angular information (a, 3) of 
the pan tilter 28 can be obtained with the positional 
coordinates (£, r|) of the desired point, the angular 
information (0, <p) representing the orientation of the 
pan tilter 28, and the magnification information (y) as 
the current zoom relative magnification assuming that 
the wide edge of the zoom lens 16 is defined as one 
magnification corresponding to Eq. (7), Eq. (8), and 
Eq. (9). 

(x, y) - <f 0 <e>, g 0 (f>) (7) 
(s, t) = (f x (Y)/ gi(Y>> (8) 
(a, p) = (f (£) , g(n) ) - . . (9) 

In Fig. 10B, positional coordinates (x, y) 
represent' the current orientation of the pan tilter 28 
assuming that the home position of « the- pan* tilter 28 is** 
defined as the origin of latitude and "longitude 
Angle-of-view information (s, t) is the current angle 
of view in the operation area 6A. Fig. 10B represents 
the states of the zoom lens and the pan tilter in the 
panorama operation area 6B. 

In Fig. IOC, angular coordinates (a, (3) are 
used to place the position designated by the pointing 
device to the center of the photographed screen 
assuming that the home position of the pan tilter 28 is 
defined as the origin of latitude and longitude. 



(PragMax, TragMax) and (PragMin, TragMin) represent the 
moving range of the pan tilter (namely, the range 
represented by the pan tilter limiter 6D) . Fig. IOC 
shows a drive target value in the pan tilter moving 
range . 

In Figs. 10A, 10B, and IOC, coordinates to be 
obtained may be absolute coordinates on the screen of 
the monitor 2 or relative coordinates assuming that the 
center of the panorama operation area 6B is defined as 
(0, 0) . In the coordinates, coordinates in the pan 
direction are represented by ml, m2, x, s, and a and 
coordinates in the tilt direction are represented by r\, 
nl, n2, y, t, and 3. 

Thus, when the mouse pointer of the mouse 8 
is present in the panorama operation area 6B, angular 
information— (a, p)-of the~pan~ tilter~28~ is' calculated 
with positional information ^,, ri) at , the mouse pointer 
of the mouse 8 corresponding to Eq. (9) - so that the 
designated object in the operation area 6A is placed at 
the center of the operation area 6A. Angular 
coordinates (a, (3) of the pan tilter 28 are converted 
into internal positional- information (PNew, TNew) of 
the pan tilter 28 corresponding to the method shown in 
Figs. 11A and 11B. The internal positional information 
(PNew, TNew) of the pan tilter 28 is stored in a send 
buffer along with an absolute position drive command of 
the pan tilter 28. In addition, as will be described 
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later, a data send request flag (FlagSo) is set so that 
data is sent upon occurrence of the timer event. 

Next, a method for converting internal 
positional information (p, t) of the pan tilter 28 into 
angular information (a, 3) and a method for converting 
angular coordinates (a, (3) into internal positional 
information (PNew, TNew) of the pan tilter 28 will be 
described with reference to Figs. 11A and 11B. In Fig. 
11A, PragMin represents angular data at the left edge 
assuming that the home position of the pan tilter 28 is 
0 (reg) . PragMax represents angular data at the right 
edge assuming that the home position of the pan tilter 
28 is 0 (rag) . PdatMin represents internal count data 
at the left edge of the pan tilter controller 25. 
PdatMax represents internal counter data at the right 
edge of the parTTil t er "c6ntfoXrer^25 . ^ "f T\ . ' r "T ^ * ' 

To obtain the pan- angle* 9-> withithe? pan- data 
p, since the following relation 1 ? is ^satlsffedif^^^y' ?" 

(PragMax - 6) : (PragMax - PragMin) = 
(PdatMax - p) : (PdatMax - PdatMin) 
the pan angle 0 is expressed as follows. 

6 = PragMax - (PragMax - PragMin) x (PdatMax 
- p) / (PdatMax - PdatMin)^ 

Thus, the pan data p is expressed as follows. 

p = PdatMax - (PragMax - 6) x (PdatMax - 
PdatMin) / (PragMax - PragMin) 

In addition, to obtain the pan data PNew with 



the pan angle <x, since the following relation is 
satisfied, 

(PragMax - a) : (PragMax - PragMin) = 
(PdatMax - p-new) : (PdatMax - PdatMin) 
the pan data PNew is expressed as follows. 

PNew = PragMax - (PragMax - a) x (PdatMax - 
PdatMin) / (PragMax - PragMin) 

In Fig. 11B, TragMin represents angular data 
at the upper edge assuming that the home position of 
the pan tilter 28 is 0 (rag) . TragMax represents 
angular data at the lower edge assuming that the home 
position of the pan tiler 28 is 0 (rag) . TdatMin 
represents internal 

count data at the upper edge of the pan 
tilter controller 25. TdatMax represents internal 
count data at the Towei^ 

c on t r 61 1 e r 2 5 . - ~w ? & **** . ■ ^.u^^^p^y^^^ „> 

To obtain" the 'tll&angi^'tf?. with . the tilt data 
t, since the following relation is satisfied, 

(TragMax - <J>) : (TragMax - TragMin) = 
(TratMax - t) : (TdatMax - TdatMin) 
the tilt angle 0 is expressed as follows. 

0 = TragMax - (TragMax - TragMin) x (TdatMax 
- t) / (TdatMax - TdatMin) 

Thus, the tilt data t is expressed as 

follows . 

t = TdatMax - (TragMax - <J>) x (TdatMax - 
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TdatMin) / (TragMax - TragMin) 

To obtain the tilt data TNew with the tilt 
angle 3, since the following relation is satisfied, 

(TragMax - (3) : (TragMax - TragMin) = 
(TdatMax - t-new) : (TdatMax - TdatMin) 
the tilt data TNew is expressed as follows. 

TNew = TragMax - (TragMax - 3) x (TdatMax - 
TdatMin) / (TragMax - TragMin) 

Next, with reference to Figs. 12A and 12B, a 
method for converting positional coordinates (£, ri) in 
the panorama operation area 6B into angular coordinates 
(a, 3) of the pan tilter 28 and a method for converting 
angular information (0, $) of the pan tilter 28 into 
positional coordinates (x, y) in the panorama operation 
area 6B will be described. In Fig. 12A, PragMin 



represents angular data at the left edge" assuming that 
the home position of f the pan tilterv 28*is* ( rag)'^#^fe*^ 
PragMax represents -angular data - at; thei right- edge*aP^P^'Xv2 
assuming that the home position of the pan tilter 28 is 
0 (rag) . Ny 2 represents a horizontal coordinate of the 
panorama operation area 6B. -Ny 2 / 2 represents 
coordinate data at the left edge of the panorama 
operation area 6B. Ny 2 / 2 represents coordinate data 
at the right edge of the panorama operation area 6B. 

To obtain the pan angle a with the coordinate 
data since the following relation is satisfied, 

(PragMax - a) : (PragMax - PragMin) = (Ny 2 / 
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2 - 5) : Ny 2 

the pan angle a is expressed as follows. 

a = PragMax - (PragMax - PragMin) x (Ny 2 / 2 

- 5) / Ny 2 

To obtain the coordinate data x with the pan 
angle 0, since the following relation is satisfied, 

(PragMax - 9) : (PragMax - PragMin) = (Ny 2 / 
2 - x) : Ny 2 

the coordinate data x is expressed as follows. 

x = Ny 2 / 2 - (PragMax - 9) x Ny 2 / (PragMax 

- PragMin) 

In Fig. 12B, TragMin represents angular data 
at the upper edge assuming that the home position of 
the pan tilter 28 is 0 (rag). TragMax represents 
angular- data^at^the~lower^ 

positicm^of., the. pan^tilter, 28 # is « O^(rag) ^ Nz 2 ^ , - 
represent sfalyerticalf coordinate ^oflrtheTp 
operation area 6B. -Nz 2 / 2 represents coordinate data 
at the upper edge of the panorama operation area 6B. 
Nz 2 / 2 represents coordinate data at the lower edge of 
the panorama operation area 6B. 

To obtain the tilt angle 3 with the 
coordinate data r\, since the following relation is 
satisfied, 

(TragMax - 3) : (TragMax - TragMin) - (Nz 2 / 
2 - n) : Nz 2 



the tilt angle 3 is expressed as follows. 

3 = TragMax - (TragMax - TragMin) x (Nz 2 / 2 

- ri) / Nz 2 

To obtain the coordinate data y with the tilt 
5 angle (}>, since the following relation is satisfied, 

(TragMax - 4>) : (TragMax - TragMin) = (Nz 2 / 
2 - y) : Nz 2 

the coordinate data y is expressed as follows. 

y = Nz 2 / 2 - (TragMax - 0) x Nz 2 / (TragMax 

10 - TragMin) 

Next, with reference to Figs. 13A to 13D, a 
method for converting angle-of-view information (\J;, co) 
captured by the pan tilter 28 into angle-of-view 
information (6, t) of the frame 6C in the panorama 
15 operation area 6B will be described. Fig. 13A shows 



the current angle-of-view information (i|r, co) of the pan 

*v - ^^#^1^^ t ilter>-28 The fangl e'r o f -v i e w -information^ WM' 1 ® is.#^>.^ 

(\Jr, co) = 1 / Y x (;|f0 f coO) ■ >.viv • -- 
20 At this point, (^0, coO) represent the horizontal angle 

of view and the vertical angle of view at the wide 
edge. A represents the magnification of the lens 
assuming that the wide edge is defined as one 
magnification. 

25 As shown in Fig. 13B, PragMin represents 

angular data at the left edge assuming that the home 
position of the pan tilter 28 is 0 (rag) . PragMax 
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represents angular data at the right edge assuming that 
the home position of the pan tilter 28 is 0 (rag) . Ny 2 
represents a horizontal coordinate of the panorama 
operation area 6B. -Ny 2 / 2 represents coordinate data 
at the left edge of the panorama operation area 6B. 
Ny 2 / 2 represents coordinate data at the right edge of 
the panorama operation area 6B. 

To obtain the horizontal angle of view s with 
the horizontal angle of view since the following 
relation is satisfied, 

\|r : (PragMax - PragMin) = s : Ny 2 
horizontal angle of view s is expressed as follows. 

s = \jr x Ny 2 / (PragMax - PragMin) 

In Fig. 13C, TragMin represents angular data 
at the lower edge assuming that the home position of 
the Tr pan"~ € i 1 1 er 28 Ts "0~(r £ g j T~^TragMax represents"^'* ^ 
angular data at the - upper- edge assuming that- the home - ^ 
position of the pan f £ilt^ 

represents a vertical coordinate of the panorama 
operation area 6B. -Nz 2 / 2 represents coordinate data 
at the lower edge of the panorama operation area 6B. 
Nz 2 / 2 represents coordinate data at the upper edge of 
the panorama operation area 6B, 

To obtain the vertical angle of view t with 
the vertical angle of view co, since the following 
relation is satisfied, 

co : (TragMax - TragMin) = t : Nz 2 



the vertical angle of view t is expressed as follows. 

t = co x Nz 2 / (TragMax - TragMin) 

Thus, the angle-of-view information (s, t) 
shown in Fig. 13D is displayed as the frame 6C in the 
panorama operation area 6B. 

Next, with reference to Fig. 14, a method for 
converting the positional information (z) of the zoom 
lens 16 into magnification information (y) will be 
described. In Fig. 14, the vertical axis represents 
information of lens magnification, whereas the 
horizontal axis represents the internal information of 
zoom lens. The positional information (z) of the zoom 
lens 16 is converted into the magnification information 
(y) by the computer 1 corresponding to a conversion 
graph shown in Fig. 14. For example, the positional 
information ( z ) ~isP converted" into" the^liT^nTficatior^ 
information (y) corresponding to a ROM * table-* or* an 
equation^, / 

Next, with reference to Fig. 15, an example 
of a control algorithm of the computer 1 will be 
described. When the program is executed, the flow 
advance to step SI. At step SI, the operation area 6, 
the panorama operation area 6B, the cursor 1, and the 
pan tilter limiter 6D are initialized and displayed on 
the monitor 2 as shown in Fig. 2. The range of the pan 
tilter limiter 6D may be fixed or variable. At step 
S2, a timer is set so that the computer 1 communicates 
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with the mode controller 23 at predetermined intervals. 
After such initial setup operations have been 
completed, the flow advances to step S3. At step S3, 
the system waits for an occurrence of an event. 
Corresponding to an event that occurs, the flow 
advances to a relevant step (for example, a timer event 
(at step S4), a mouse button down event (at step S5) , a 
mouse button up event (at step S6) , and a mouse move 
event (at step S7) ) . 

Next, with reference to a flow chart shown in 
Figs. 16A and 16B, the algorithm of the timer event 
will be described. The timer event is an event for 
causing the computer 1 to communicate with the mode 
controller 23 at predetermined intervals. The timer 
event occurs at intervals. of for example 50 msec. When 
the timer event occurs^ the" flow advances ~tb^step SllT " ^ 
At step , Sll the . system, determines whether ,or, not.a-.^,^... 
communication^ port has been set . When the' W?*?^. >vf!S^i . 
communication port has been set (namely, the determined 
result at step Sll is Yes), the flow advances to step 
S12. When the communication port has not been set 
(namely, the determined result at step Sll is No) , the 
flow advances to step Sl8. ? At the first time the 
communication port has not been set, the flow advances 
to step S18. At step S18, the system opens the 
communication port. Actually, at step S18, the system 
opens an RS-232C port of the computer 1. Thereafter, 

•>. 

38 



the flow advances to step SI 6. 

Thereafter, in the timer event, the system 
performs a receive data checking process, an analyzing 
process, a data sending process for data stored in the 
send buffer (such as the drive command for the pan 
tilter 28), and/or a communication data sending process 
for state check requests for the pan tilter 28 and the 
zoom lens 16. In this algorithm, the flow advances 
from step Sll to step S12. At step S12, the system 
determines whether or not data is stored in the receive 
buffer. When data is stored in the receive buffer 
(namely, the determined result at step S12 is Yes), the 
flow advances to step S13. When data is not stored in 
the receive buffer (namely, the determined result at 
step S12 is No), the flow advances to step S14. At 
^ step~S13,~ the^sys tern 7 analyzes^ receive data "stored in ^ 
•4 receive buffer , and obtains^ positional^ information 
^1 ^ ( P ' t > - ° %\ ^he? Pan : tilter 28; and^f positional- information 
(z) of the zoom lens 16 that have been requested to the 
mode controller 23. The system converts the positional 
information (P/ t) of the pan tilter 28 and the 
positional information (z) of the zoom lens 16 into 
angular information (6, 0)- of the pan tilter 28 and 
magnification information (y) of the zoom lens 16 
corresponding to methods shown in Fig. 11 and 14. 

At step S14, the system determines whether or 
not a data send request has been issued. When a data 



send request has been issued (FlagSo == True) (namely, 
the determined result at step S14 is Yes), the flow 
advances to step S19. At step S19, the system sends 
data stored in the send buffer and resets the send 
request flag (FlagSo == False). Next, the flow 
advances to step S16. An example of data stored in the 
send buffer is data of a drive command of the pan 
tilter 28 designated with the mouse 8. When a send 
request has not been issued (FlagSo == False) (namely, 
the determined result at step S14 is No) , the flow 
advances to step S15. At step S15, the system sends 
position request commands for the pan tilter 28 and the 
zoom lens 16 from the computer 1 to the mode controller 
23. 

At step S16, the system compares the old 
pbsitional^infbi^ati^ t i 1 e r~~2 8"^wi tlT" t He "ii'e W^T^' 

^positionalvinformation, thereof^and . determines* whe t he riW&^h 
fSt or: not* the" positional* information*: • (^ t )|fha^^axied 
When the positional information (p, t) of i the pan 
tilter 28 has varied (namely, the determined result at 
step S16 is Yes), the flow advances to step S20. When 
the positional information (p, t) of the pan tilter 28 
has not varied (namely, -the determined result at step 
S16 is No), the flow advances to step S17. At step 
S17, the system compares the old positional information 
of the zoom lens 16 with the new positional information 
thereof and determines whether or not the positional 
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information (z) has varied. When the positional 
information (z) of the zoom lens 16 has varied (namely, 
the determined result at step S17 is Yes), the flow 
advances to step S20. When the positional information 
(z) of the zoom lens 16 has not varied (namely, the 
determined result, at step S17 is No) , this event is 
completed. 

At step S20, when the positional information 
(p, t) of the pan tilter 28 and/or the positional 
information (z) of the zoom lens 16 has varied, the 
system redraws the frame 6C in the panorama operation 
area.6B. At this point, the system converts the 
positional information (p, t) of the pan tilter 28 into 
the angular information (0, <J>) . In addition, the 
system converts the positional information (z) of. the 
zoom lens " 1"6 ^ilnto^the " magnificat 
„. With the. conve rted~ r angular * irif ormat ion* ( 9 / $ ) « and • • ' 
'magnification! information ? ( " Y 1W tffPsys tern calculates^ ' 
positional coordinates (x, y) of the pan tilter 28 and 
angle-of-view information (s, t) that is the angle of 
view displayed in the operation area 6A corresponding 
to Eq. (7) and Eq. (8), respectively. Corresponding to 
the resultant positional coordinates (x, y) and angle- 
of-view information (s, t) , the system draws the frame 
6C in the panorama operation area 6B. 

At step SI 6, the system compares the old 
positional information (p, t) of the pan tilter 28 with 



the new positional information (p, t) thereof. 
Alternatively, the system may compare the old angular 
information (0, <j>) of the pan tilter 28 with the new 
angular information (0, cf>) thereof. In this case, at 
step S20, with the new angular information (0, <j>) , the 
system calculates the positional coordinates (x, y) 
corresponding to Eq. (7) . Likewise, at step S17, the 
system compares the old positional information (z) of 
the zoom lens 16 with the new positional information 
(z) thereof. Alternatively, the system may compare the 
old magnification information (y) of the zoom lens 16 
with the new magnification information (y) thereof. In 
this case, at step S20, the system calculates the 
angular information (s, t) with the new magnification 
information (y) corresponding to Eq. (8) . 

^^F±g.^ll ,^the ; i algorithm.! of the, mouse* move| events will be 

occurs when the mouse 8 is moved. According to the 
present invention, the mouse move event is used to 
select a drive position of the pan tilter 28. When the 
mouse move event occurs, the flow advances to step S21. 
At step S21, the system determines whether or : not the 
mouse pointer of the mouse 8 is present in the 
operation area 6A, the panorama operation area 6B, or 
the other area. When the mouse pointer of the mouse 8 
is present in the operation area 6A (namely, the 
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determined result at step S21 is Yes), the flow 
advances to step S22 . When the mouse pointer of the 
mouse 8 is not present in the operation area 6A 
(namely, the determined result at step S21 is No), the 
flow advances to step S24. At step S22, the system 
sets an operation area flag (Flag-rin == True) and 
clears a panorama operation area flag (Flag-pin === 
False) . 

At step S24, since the mouse pointer of the 
mouse 8 is not present in the operation area 6A, the 
system clears the operation area flag (Flag-rin == 
False) . At step S25, the system determines whether or 
not the mouse pointer of the mouse 8 is present in the 
panorama operation area 6B. When the mouse pointer of 
15 the mouse 8 is present in the panorama operation area 
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" \'. •"' — - 6B- (namelyr* the~determined"result^ Yes )', T 0 *** 

f low advances^ to - step S2 6 .^W^n^the ; mouse - pointer *>^: 
^Afe^ ^#>f- the ii riouse >^ 8| isfnotS present|lnftl^f panorama operation" 
area 6B (namely, the determined result at step S25 is 
20 No), the flow advances to step S27. At step S26, the 

system sets the panorama operation area flag (Flag-pin 
* == True) . At step S27, since the mouse pointer of the 

mouse 8 is not present in the panorama operation area 
6B, the system clear the panorama operation area flag 
25 (Flag-pin == False) . 

When the mouse pointer of the mouse 8 is 
present in the operation area 6A or the panorama 
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operation area 6B (namely, the determined result at 
step S21 or step S25 is Yes), at step S23, the system 
obtains positional coordinates (£, ri) of the mouse 
pointer of the mouse 8 assuming that the center of the 
5 operation area is defined as (0, 0) of relative 

coordinates . 

In this flow chart, when the mouse pointer of 
the mouse 8 is present in the operation area 6A 
(namely, the determined result at step S22 is Yes), the 
10 system sets the operation area flag (Flag-rin == True) . 

When the mouse pointer of the mouse 8 is not present in 
the operation area 6A (namely, the determined result at 
step S22 is No) , the system clears the operation area 
flag (Flag-rin == False) . When the mouse pointer of 

15 the mouse 8 is present in the panorama operation area 

' ; ^ • ' 6B-(namely7^the^determined^result~at step S25~is Y&)7*T K TT* 

^.a^^ panorama operation- area^flag* (Flag- 

; ' W^ll^f ' P in ;i === True ) When the? mouse? pointer:* 8^istnot! present i 
in the panorama operation area 6A (namely, the 

20 determined result at step S25 is No) , the system clears 

the panorama operation area flag (Flag-pin == False) . 
* When the mouse pointer of the mouse 8 is present in the 

operation area 6A or the- panorama operation area 6B 
(namely, the determined result at step S21 or S35 is 

25 Yes), the system designates the positional coordinates 

of the mouse pointer of the mouse 8 to (£, ri) assuming 
that the center of each operation area is defined as 



(0, 0) of relative coordinates. 

Next, the mouse button down event and the 
button up event will be described. In the method for 
directly designating a desired point of the operation 
area 6A or the panorama operation area 6B, only the 
algorithm of a mouse button down event shown in Fig. 18 
is used. In the method for designating a desired point 
generated with a desired area, both the algorithm of a 
mouse button down event shown in Fig. 19 and the 
algorithm of a mouse button up event shown in Fig. 20 
are used. 

With reference to a flow chart shown in Fig. 
18, the algorithm of the button down event for the 
method for directly designating a desired point of the 
operation area will be described. This event is an 
event— that- occurs- when- the lef t^button" of ^the* mouse <r 8 T ^ 
i ? P£® s 5®5*% . In present invent ion, ^his^even^ 

US ?^X**$:^ fori driving| the; pan^tilter 

28. When this event occurs, the flow advances to step 
S31. At step S31, the system determines whether or not 
the mouse pointer 8 is present in the operation area 6A 
corresponding to the operation area flag. When the 
operation area flag has been set (FlagRin == True) 
(namely, the determined result at step S31 is Yes), 
since the mouse pointer of the mouse 8 is present in 
the operation area 6A, the flow advances to step S32. 
When the operation area flag has been cleared (FlagRin 



~ False) (namely, the determined result at step S31 is 
No), since the mouse pointer of the mouse 8 is not 
present in the operation area 6A, the flow advances to 
step S34 . 

When the mouse pointer of the mouse 8 is 
present in the operation area 6A (namely, the 
determined result at step S31 is Yes), the flow 
advances to step S32. At step S32, the system 
calculates angular information (a, p) of the pan tilter 
28 with the angular information (9, (J>) of the current 
pan tilter 28 obtained from the received data, the 
magnification information (y) of the zoom lens 16, and 
the positional coordinate (9, rj) of the mouse pointer 
of the mouse 8 in the operation area 6A corresponding 
to Eq. (4) or Eq. (5) so that the designated object in 
.the^o^ a t" : f tKetcen t e r 

^ At ^ s t ep ? S 3 3 ;%£ lief "s ys £e^^wtrt's the ' angular 
information (a, 3) of the pan tilter 28 into the 
internal positional information (PNew, TNew) 
corresponding to the method shown in Fig. 11. The 
system stores the converted positional information 
(PNew, TNew) in the send buffer along with the absolute 
position drive command of the pan tilter 28. In 
addition, the system sets the data send request flag 
(FlagSo == True) and sends the data with the process of 
the timer event. 



After the system has determined that the 
mouse pointer of the mouse 8 is not present in the 
operation area 6A (namely, the determined result at 
step S31 is No), the flow advances to step S34. At 
step S34, the system determines whether or not the 
mouse pointer of the mouse 8 is present in the panorama 
operation area 6B corresponding to the panorama 
operation area flag. When the panorama operation flag 
has been set (FlagPin == True) (namely, the determined 
result at step S34 is Yes), since the mouse pointer of 
the mouse 8 of the panorama operation area 6B is 
present in the panorama operation area 6B, the flow 
advances to step S35. When the panorama operation flag 
has been cleared (FlagPin == Fale) (namely, the 
determined result at step S34 is No), this event is 
cbmp 1 e tVd y* * 1 **^^ ^^ AAU> ww**?*?***^ :j - • - : ■" ' T 

■8*%^^ determines - 

whether^ of vtheTmouseV 8 is : - 

present in the operation area 6A or the panorama 
operation area 6B corresponding to the operation area 
flag (FlagRin) and the panorama operation area flag 
(FlagPin) . When the mouse pointer of the mouse 8 is 
not present in the operation area 6A and the panorama 
operation area 6B, this event becomes invalid. 

When the mouse pointer of the mouse 8 is 
present in the panorama operation area 6B (namely, the 
determined result at step S34 is Yes), the flow 
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advances to step S35. At step S35, the system 
calculates angular information (a, (3) of the pan tilter 
28 with the positional information (£, ri) at the mouse 
pointer of the mouse 8 in the panorama operation area 
6B corresponding to Eq. (9) so that the designated 
object in the operation area is placed at the center of 
the screen. Thereafter, the flow advances to step S33. 

Next, with reference to Figs. 19 and 20, the 
algorithms of the button down event and the button up 
event for the method for designating a desired point 
generated with a desired area in the panorama operation 
area 6B will be described, respectively. 

With reference to the flow chart shown in 
Fig. 19, the algorithm of the button down event will be 
described. This event is an event that occurs when the 
le f t^button^ of ^ the mous e ~ 8~ i s~ p r e s's edT^ In this 
e^pdiment,4.this_event^ is vi i used^as, an, event, for 
determining! the v starts point off a" desired area : 1 When 
this event occurs, the flow advances to step S41. At 
step S41, the system determines whether or not the 
mouse pointer of the mouse 8 is present in the 
operation area 6A corresponding to the operation area 
flag (FlagRin) . When the operation area flag has been 
set (FlagRin == True) (namely, the determined result at 
step S41 is Yes), since the mouse pointer of the mouse 
8 is present in the operation area 6A, the flow 
advances to step S42. When the operation area flag has 
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been cleared (FlagRin == False) (namely, the determined 
result at step S41 is No), since the mouse pointer of 
the mouse 8 is not present in the operation area 6A, 
the flow advances to step S44. 

When the mouse pointer of the mouse 8 is 
present in the operation area 6A (namely, the 
determined result at step S41 is Yes), at step S42, the 
system sets an operation area start point obtain flag 
(FlagRstart = True) . Thereafter, the flow advances to 
step S43. At step S43, the system stores positional 
coordinates (ml, nl) at which the left button of the 
mouse 8 is pressed as the start point of the desired 
area. 

After the system has determined that the 
mouse pointer of the mouse 8 is not present in the 
^toperation-ar e a* 6AT^a t^s pep* S 4 47^tKe^yst 
;Si^ w ^ th ^ 8 -is 

!^^P r ^s^ntWintthe^>anorama^ operationfaree?? 6B ' corresponding 
to the panorama operation area .flag (FlagPin) . When 
the panorama operation area flag has been set (FlagPin 
■-■•==■ True) (namely, the determined result at step S44 is 
Yes), since the mouse pointer of the mouse 8 is present 
in the panorama operation area 6B, the flow advances to 
step S45. When the panorama operation area flag has 
been cleared (FlagPin == False) (namely, the determined 
result at step S44 is No), this event is completed. 

In this flow chart, the system determines 



whether or not the mouse pointer of the mouse 8 is 
present in the operation area 6A or the panorama 
operation area 6B corresponding to the operation area 
flag (FlagRin) and the panorama operation area flag 
5 (FlagPin) . When the mouse pointer of the mouse 8 is 

not in the operation area 6A and the panorama operation 
area 6B, this event becomes invalid. 

When the mouse pointer of the mouse 8 is 
present in the panorama operation area 6B (namely, the 
10 determined result at step 44 is Yes) , the flow advances 

to step S45. At step S45, the system sets a panorama 
operation area start point obtain flag (FlagPstart) . 
Thereafter, the flow advances to step S43. 

Next, with reference to a flow chart shown in 
15 Fig. 20, the algorithm of the button up event will be 

-'•^describedf^This- event? is*ari~ event ^thatr^occurs^wKerr the 



event " for ; 

determining the end point of a desired area. 

20 When this event occurs, the flow advances to 

step S51. At step S51, the system determines whether 
■ or not the operation area flag has been set (FlagRin === 

True) (namely, the mouse pointer of the mouse 8 is 
present in the operation area 6A) . When the operation 

25 area flag has been set (FlagRin = True) (namely, the 

determined result at step S51 is Yes), since the mouse 
pointer of the mouse 8 is present in the operation area 



50 



6A, the flow advances to step S52 . When the operation 
area flag has been cleared (FlagRin == False) (namely, 
the determined result at step S51 is No) , since the 
mouse pointer of the mouse 8 is not present in the 
operation area 6A, the flow advances to step S57. At 
step S52, the system determines whether or not the left 
button of the mouse 8 has been pressed in the operation 
area 6A corresponding to an operation area start point 
obtain flag (FlagRstart) . When the start point obtain 
flag has been set (FlagRstart == True) (namely, the 
determined result at step S52 is Yes), since the left 
button of the mouse 8 has been. pressed in the operation 
area 6A, the flow advances to step S53. When the start 
point obtain flag has been cleared (FlagRstart ™ 
False) (namely, the determined result at step S52 is 
No)7Tsince~ the^lef t *• buttoi^ been 
pressed^in^the^ operation|areai6A,^the^ flow-advances to 

In other words, at steps S51 and S52, the 
system determines whether or not the operation area 
flag and the operation area start point obtain flag 
have been set or cleared. , When the operation area flag 
and the start point obtain flag have been set (FlagRin 
== True and FlagRstart == True), the system determines 
that the drive command has taken place in the operation 
area 6A. Otherwise, at steps S57 and S58, the system, 
determines whether or not the panorama operation area 
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flag (FlagPin) and the panorama operation area start 
point obtain flag (FlagPstart) have been set or 
cleared. 

When the drive command has taken place in the 
operation area (namely, the operation area flag and the 
start point obtain flag have been set (FlagRin == True 
and FlagRstart == True), at step S53, the system stores 
the positional coordinates (m2, n2) at which the left 
button of the mouse 8 has been released in the 
operation area 6A as the end point of the desired area. 
Thereafter, the system calculates positional 
information (£, nj as the coordinates of the center of 
the rectangle area generated with the positional 
coordinates (ml, nl) of the start point of the desired 
area and the positional coordinates (m2, n2) of the end 
point"" thereof " ' ^"■ L - , ' ,m " '■ *»' wmmmpjwm* ■■- .. mvm*&* n *r r ~ , + *•• -• - 

i*^^ Ats* step.; S54 the>" system calculates angular 
in^6rmation|:(a^^ 28 with the 

angular information (0, <j>) of the pan tilter obtained 
from the received data, the magnification information 
(Y) of the zoom lens 16, and the positional information 
(€, n) at the mouse pointer of the mouse 8 
corresponding to Eq. (4)- or Eq. (5) . 

At step S55, the system converts the angular 
information (a, p) of the pan tilter 28 into the 
internal positional information (PNew, TNew) of the pan 
tiler 28 corresponding to the method shown in Fig. 11 
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and stores the positional information (PNew, TNew) to 
the send buffer along with the absolute position drive 
command. In addition, the system sets the data send 
request flag (FlagSo == True) and sends data with the 
5 process of the timer event. 

At step S56, after the system has checked the 
mouse button up event in each operation area, the 
system clears the operation area start point obtain 
flag and the panorama operation area start point obtain 

10 flag (FlagRstart == False and FlagPstart == False) . 

Thereafter, this event is completed. 

At step S57, the system determines whether or 
not the mouse pointer of the mouse 8 is present in the 
panorama operation area 6B corresponding to the 

15 panorama operation area flag (FlagPin) . When the 



. _ —^panorama "operation areaL flag- has' beeh™set~( FiagPi7T==^ 
,^4^ at - stepcSST- i 

'''^T^k?*'*) ,f since t the'' mouset point'e^of | the'" mousey 8| is - ' pres ent " 
in the panorama operation area 6B, the flow advances to 

20 step S58. When the panorama operation area flag has 

not been set (FlagPin == False), since the mouse 
* pointer of the mouse 8 is not present in the panorama 

operation area 6B, the flow advances to step S56. At 
step S58, the system determines whether or not the left 

25 button of the mouse 8 has been pressed in the panorama 

operation area 6B corresponding to the panorama 
operation area start point obtain flag (FlagPstart) . 
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When the start point obtain flag has been set 
(FlagPstart == True) (namely, the determined result at 
step S58 is Yes), since the left button of the mouse 8 . 
has been pressed in the panorama operation area 6B, the 
flow advances to step S59. When the start point obtain 
flag has not been set (FlagPstart == False) (namely, 
the determined result at step S58 is No) , since the 
left button of the mouse 8 has not been pressed in the 
panorama operation area 6B, the flow advances to step 
S56. 

When the panorama operation area flag and the 
panorama operation, start point obtain flag have been 
set (FlagPin == True and FlagPstart == True) at steps 
S57 and S58, the system determines that a drive command 
has issued in the panorama operation area 6B. When the 
conditionsTat^s teps^S5 1 , ' S52 ;~ana~S58~are^not v ^ " " 

H|^PPI||^^ ; When; the : drive command? has- been issued in the 
panorama operation area 6B (namely, the panorama 
operation area flag and the start obtain flag have been 
set (FlagPin == True and Flag-pstart == True), the flow 
advances to step S59. At step S59, the system stores 
the positional coordinates 1 (m2, n2) at which the left 
button of the mouse 8 has been released in the panorama 
operation area 6B as the end point of the desired area. 
The system calculates the positional information (£, n) 
of the mouse pointer of the mouse 8 as the coordinates 
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of the center of the rectangle area with the positional 
coordinates (ml, nl) of the start point of the desired 
area that has been stored and the positional 
coordinates (m2, n2) of the end point of the desired 
area corresponding to Eq. (6) . 

At step S60, the system calculates angular 
information (a, p) of the pan tilter 28 with the 
positional information (£, r\) at the mouse pointer of 
the mouse 8 in the panorama operation area 6B 
corresponding to Eq. (9) so that the designated object 
in the panorama operation area is placed at the center 
of the screen. Thereafter, the flow advances to step 
S55. 

In the above-described embodiment, one 
computer performs all processes of the system. On the 
fother-handT^accordin^ ttf^^ of the ~ 

present ,invention,^as^shown|in^Fig.^ 21, processes are 
^ h ? red i b y^ a server compute computer so as 

to control a pan tiler camera through a network that 
has a restriction of a communication capacity. In Fig. 
21, a computer 1 is connected to a monitor 2 and a 
mouse 8. The computer 1- controls the operation of a 
pan tilter camera 3 disposed at a remote place through 
a transmission line and a server 9. In other words, 
the computer 1 composes a controller for a 
photographing apparatus. The transmission line may be 
a communication line (radio communication line or a 
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cable communication line), a network, or the like. The 
computer 1 has a relation of a client to the server 9. 
A plurality of computers 1 can be connected to the 
server 9. 

The pan tilter camera 3 and the server 9 are 
disposed on a real scene in an environment denoted by 
reference numeral 4. A screen photographed by the pan 
tilter camera 3 disposed on the real scene 4 is denoted 
by reference numeral 5. Hereinafter, the screen 5 is 
referred to as photographed screen. The photographed 
screen 5 is an actually photographed screen. When the 
zoom lens is placed on the telephotograph side, the 
angle of view decreases. In contrast, when the zoom 
lens is placed on the wide-angle side, the angle of 
view increases. 

^^^^^^ A -- p i ctur e photographed by the parT tilter 7 
caiiiera A 5^ is;. sent^to^a^server^9^|^The^ server., 9 . converts 
;the photographed) picture^ intc^ideo "data f- % The? video . ■ 
data is sent to the computer 1 through a transmission 
line. The video data sent to the computer 1 is decoded 
and displayed on the monitor 2. The monitor 2 displays 
the photographed screen 5 in the operation area 6A 
thereof. A panorama picture with which a picture 
photographed by the pan tilter camera 3 is superimposed 
is displayed in the panorama operation area 6B. As 
with the above-described embodiment, a desired point of 
the panorama operation area 6B (or the operation area 
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6A) or a desired point generated with a desired point 
is designated with the mouse 8 (cursor 7) . The pan 
tilter camera 3 is driven through the server 9 and the 
transmission line and thereby the photographed screen 
5 is moved. In other words, the pan tilter camera 3 is 

controlled through the server 9 so that the selected 
object is placed at the center of the operation area 
6A. 

Fig. 22 is a block diagram showing the 
10 overall system of the second embodiment of the present 

invention. Since the structures and functions of the 
camera portion 11 and the pan tilter portion 12 are the 
same as those of the first embodiment, the structures 
thereof are omitted in Fig. 22. The server 9 comprises 
15 a controlling portion 131, a video capture portion 129, 

^^'and^a* storing -portion" 130T*^ The" -video^'capture portion 
^^^^12 ^is^ : composed, of . a. .video^ capture* board .uh The computer 

%P^^^ a : ' 

'. network. The computer 1 is composed of a controlling 

20 portion 31 and so forth as with the first embodiment. 

Since the algorithms used in the computer 1 are the 
+ same as those of the first embodiment, for simplicity, 

their description is omitted. 

Rays emitted from an object are sent to the 
25 camera portion 11 as with the first embodiment. The 

camera portion 11 converts the rays into various 
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signals such as a brightness signal (Y) , a color signal 
(C) , and a video signal and supplies the resultant 
signals as picture signals to a TV monitor 13 and the 
video capture portion 129 of the server 9. As with the 
first embodiment, the pan tilter portion 12 has a mode 
controller, a camera controller, and a pan tilter 
controller. These controllers control the camera 
portion 11 and the pan tilter 28. The mode controller 
23 controls the overall system corresponding to the 
internal states of the camera portion 11 and the pan 
tilter portion 12 and an external command as with the 
first embodiment. 

The mode controller 23 is connected to the 
server 9 through a communication path (in reality, 
RS232C interface). The mode controller 23 sends 
commands received" f rom th^server" 9 and commands' ~ 
^^^Y^A . f rom^the^computer^ 1* through* - the server 9 to - 
pan j tilter^cprit roller. andftlie|camera controller so 
; as to drive the pan tilter and the zoom lens of the 
lens block portion. The mode controller 23 always 
receives information from the pan tilter controller and 
the camera controller so- as to send the inner state of 
the pan tilter camera to^ the outside through the server 
9. 

The server 9 obtains the inner state of the 
pan tilter camera (for example, the current positional 
information of the pan tilter and the zoom lens, and so 
forth) from the mode controller 23 of the pan tilter 



portion 12 at predetermined intervals . To send a 
picture photographed by the camera portion 11 to the 
transmission path 132, the video capture portion 129 is 
used. The video capture portion 129 converts a picture 
signal received from the camera portion 11 to digital 
picture data that is sent to the transmission path 132 
in any quality (in the present embodiment, still 
picture JPEG format or still picture bit map format) . 
The resultant digital picture is stored in a storing 
portion 130 (for example, a hard disk) . 

When the computer 1 issues a connection 
request to the server 9, the server 9 sends a GUI 
(Graphical User Interface) panel information to the 
computer 1 so as to display a picture on the monitor 2. 
The panel information is an arrangement of a panel and 
ar program- that^runs^on^ the "computer 1' when the mouse "is 
^Operated ^qn ..the panel/^^Examples. of the panel ... 
information* aref programs^ written in HTML, JAVA, and so 
forth. Picture data photographed by the pan tilter 
camera and the state thereof are sent to the computer 1 
through the transmission path 132 at predetermined 
intervals. 

In another embodiment, Internet is used as 
the transmission path 132. Data is exchanged on the 
transmission path 132 using the HTTP protocol. The 
computer 1 causes the monitor 2 to display GUI panel 
information, picture information, the state of the pan 
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information, picture information, the state of the pan 
tilter camera, and so forth received from the server 9 
with an Internet browser. An operation area 6A, a 
panorama operation area 6B, a panorama picture 
generation button 6E, zoom operation buttons, a cursor 
7 of a pointing device 14 (mouse 8), and so forth 
displayed on the GUI panel of the monitor 2. Picture 
data received from the server is decoded and displayed 
in the operation area 6A. When the picture data is 
updated, the picture is also rewritten in the operation 
area 6A. The moving range of the pan tilter camera, 
the position of the pan tilter, angle-of-view of the 
zoom, and so forth are displayed on the panorama 
operation area 6B, with the same method as the first 
embodiment. The computer 1 executes the operation 
program~f or^the^GUI^panel T received from the server 9 . ~ 
1$$^^ e cond :> embodiment of the present 

inyention^a|^drive rcommand^of ? the pan tilter camera 3 
and an operation command of the server 9 are generated 
with a clicking operation of the mouse 8. When the 
mouse 8 is clicked on the panorama generation button 
6E, the computer 1 causes the server 9 to generate a 
panorama screen. When the-^ server 1 receives this 
command, as with the first embodiment, the server 9 
moves the pan tilter and the zoom lens to relevant 
positions, photographs ten pictures at these positions, 
maps them to the virtual spherical surface, normalizes 
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them with latitude and longitude, and combines them. 
After the server 9 has combined these pictures as a 
panorama picture, it converts the panorama picture into 
a JPEG format picture. The servers 9 sends the 
resultant picture to the computer 1 through the 
transmission line 132. 

The computer 1 displays the received panorama 
picture in the panorama operation area 6B of the 
monitor 2. Thus, the user can see the environment at 
the position of the pan tiler camera 3 at a glace. 
When the mouse 8 is clicked in the panorama operation 
area 6B, the computer 1 sends to the server 9 a command 
(absolute position drive command) that causes the 
position at which the mouse is clicked on the panorama 
picture to be placed at the center of the operation 
area 6A f (picture YT** The: server 9 "sends ^this^command to ' 
the,pan;,ti^ tilter is 

driven? to^a^elevantrpbsition^ In such a manner, the 
drive target of the pan tilter is designated on the 
panorama screen. Consequently, the user can easily 
operate the pan tilter without need to consider a drive 
command on the network, a delay of a video signal, and 
so forth. • 

In the first embodiment, whenever the pan 
tilter camera 3 sends a picture to the computer 1, the 
computer 1 combines it and displays the combined 
picture in the panorama operation area 6B. 
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Alternatively, after the computer has combined all 
pictures, it may display the resultant picture in the 
panorama operation area 6B. 

According to the first embodiment, the 
operation area 6A and the panorama operation area 6B 
are displayed on the monitor 2 connected to the 
computer 1. Alternatively, the operation area 6A 
and/or the panorama operation area 6B may be displayed 
on another display unit other than the monitor 2. 

According to the first embodiment, the pan 
tilter camera 3 is driven by operating the operation 
area 6A and the panorama operation area * 6B with the 
mouse 8 . Alternatively, one of the operation area 6A 
and the panorama operation area 6B may be operated with 
the mouse 8. 

Accordihg~to" the" f irst^embodimeht ; the 

panorama operation area 6B 

# ■ V ■& r &-.i%[&%&**'< —V - A ' fe'- '\&^M; ^'<' : 'V. . 

^ 2 ••^Alternatively, only 

the panorama operation area 6B may be displayed on the 
monitor 2. 

According to the first embodiment, the 
operation area 6A and the panorama operation area 6B 
are displayed on the monitor 2. By operating the 
operation area 6A and the panorama operation area 6B 
with the mouse 8, the pan tilter camera 3 is freely 
driven. Alternatively, a panorama picture may be 
displayed on the monitor 2. In this case, the pan 
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tilter camera 3 may be driven with an operation portion 
such as eight-direction keys. 

According to the above-described embodiments, 
the photographing range of the pan tilter camera 3 may 
be the maximum moving range of the pan tilter camera 3 
or limited with a limiter. The function for limited 
the photographing range with the limiter may be 
provided by the pan tilter camera 3 or the computer 1. 

In the first embodiments, a desired point 
generated with a desired area is placed at the center 
of thereof. Alternatively, a desired point may be 
placed at for example the center of gravity, the 
incenter, the circumcenter, or the orthocenter of the 
area. 

According to the first embodiment, a panorama 
picture^di splayed" in tEe^p^ area 6B is ~ ~ ~ 

not^li^te^ the environment in^l^V-"' 4 ' - 

whicl^h^pa^ For y n'M0^f^<y 

example, i the panorama picture may be a moving picture, 
an intermittent still picture, or a still picture. 

According to the second embodiment, for 
simplicity, one computer 1 is connected to the remote 
server 9 and the pan tilter camera 3 that are : disposed 
at a remote place. Alternatively, a plurality of 
servers 9 and a plurality of pan tilter cameras 3 may 
be disposed worldwide. For example, one pan tilter 
camera 3 may be controlled by a plurality of computers 
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through for example Internet. 

According to the present invention, with a 
panorama picture, the user can see the environment in 
which the photographing apparatus is disposed at a 
glance. Since the positional information of the pan 
tilter, the angle of view of the zoom lens, and the 
moving range of the pan tilter are added as information 
to the picture, the user can easily know the state of 
the photographing apparatus. 

In addition, when the user designates a 
desired object in the panorama operation area, he or 
she can easily capture it in the field of view of the 
picture to be photographed. Moreover, by designating 
an object in the operation area, the user can precisely 
adjust the position that cannot be designated in the 
pahdraifa' " opeFatl^bn^rea ♦ ^ ' in" comparison with the" 
conventional method* of &whichvthe* user- operates *•■*;•• ,,,>.,.,>, ^^..>. 

direction* keys^ whiles observingfa^ monitored picture 
(namely, a picture is photographed through a feed-back 
operation and an experience) , according to the present 
invention, a desired object can be displayed at the 
center of the operation area with the clicking 
operation of the mouse. • • 

In addition, according to the present 
invention, since the position to which the pan tilter 
moves can be predicted beforehand, on a communication 
line that causes picture and information data to be 

■-. . 
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delayed and/or lost (such as Internet), the user can 
seamlessly operate the pan tilter camera. Thus, 
according to the present invention, the pan tilter 
camera can be easily operated with high visibility. 
5 Although the present invention has been shown 

and described with respect to a best mode embodiment 
thereof, it should be understood by those skilled in 
the art that the foregoing and various other changes, 
omissions, and additions in the form and detail thereof 
10 may be made therein without departing from the spirit 

and scope of the present invention. 
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